feat(event cache): automatic back-pagination for unread counts, MVP#6344
feat(event cache): automatic back-pagination for unread counts, MVP#6344
Conversation
…sing a sync lock There wasn't good reason to use an async lock, as this lock is always super short-lived, it can be sync, which avoids complications in the subsequent commit when calling sync init code.
This makes it simpler to test its expected behavior, and will make it trivial to add a system to request automatic paginations in the background.
…has been found when computing unread counts
…ng a room with missing receipts
I would personally go the other way: first address this problem, because it's not an easy one. It could be part of the |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #6344 +/- ##
========================================
Coverage 89.85% 89.85%
========================================
Files 376 376
Lines 102778 102931 +153
Branches 102778 102931 +153
========================================
+ Hits 92352 92493 +141
- Misses 6859 6870 +11
- Partials 3567 3568 +1 ☔ View full report in Codecov by Sentry. |
I think it's actually less pragmatic to solve this hard problem now, because by having something available behind a feature flag that can be toggled dynamically at runtime, a feature which is already implemented in this PR, we can inform ourselves about what is missing in this implementation and learn otr more about the requirements and see how much of a problem in practice it is, before engineering a complex solution that may not be useful after all. (Note the solution isn't as simple as "do only in store vs do in-memory", because a subscription might happen while we're in the "do only in store mode".) |
2 participants
This adds a very basic system for automatic back-paginations, with a way to trigger background requests from any component. At the moment it's super simple: it will only run one pagination request at a time, based on a maximum number of pagination credits per room (defaulted to 20). It's disabled by default, and there's a config toggle to enable it at the
EventCachelevel; this toggle must be set before subscribing to the event cache (in particular, for consumers of the FFI layer, that means before creating a timeline or a room list service or a sync service).One open question: this may leave room event caches in a state where a linked chunk has many events loaded in memory. This might be an issue, but since this is experimental, I'd rather land this piecemeal, rather than having the perfect implementation at start (because shrinking the linked chunk would cause paginations to happen just thereafter, if a read receipt points to an event that is far away enough that it'd be only loaded on disk, thereby causing an infinite loop of shrinking / reloading / shrinking etc.). Is that fine, for the moment?
Most of #6014.
CHANGELOG.mdfiles.